The present invention relates to entropy coding and may be used in applications such as, for example, video and audio compression.
The present invention describes a new method and apparatus for entropy encoding and decoding of discrete data. Entropy coding, in general, can be considered as the most generic form of lossless data compression. Lossless compression aims to represent discrete data with fewer bits than needed for the original data representation but without any loss of information. Discrete data can be given in the form of text, graphics, images, video, audio, speech, facsimile, medical data, meteorological data, financial data, or any other form of digital data.
In entropy coding, the specific high-level characteristics of the underlying discrete data source are often neglected. Consequently, any data source is considered to be given as a sequence of source symbols that takes values in a given m-ary alphabet and that is characterized by a corresponding (discrete) probability distribution {p1, . . . , pm}. In these abstract settings, the lower bound of any entropy coding method in terms of expected codeword length in bits per symbol is given by the entropy
                    H        =                  -                                    ∑                              i                =                1                            m                        ⁢                                          p                i                            ⁢                              log                2                            ⁢                                                p                  i                                .                                                                        (        A1        )            
Huffman codes and arithmetic codes are well-known examples of practical codes capable of approximating the entropy limit (in a certain sense). For a fixed probability distribution, Huffman codes are relatively easy to construct. The most attractive property of Huffman codes is that its implementation can be efficiently realized by the use of variable-length code (VLC) tables. However, when dealing with time-varying source statistics, i.e., changing symbol probabilities, the adaptation of the Huffman code and its corresponding VLC tables is quite demanding, both in terms of algorithmic complexity as well as in terms of implementation costs. Also, in the case of having a dominant alphabet value with pk>0.5, the redundancy of the corresponding Huffman code (without using any alphabet extension such as run length coding) may be quite substantial. Another shortcoming of Huffman codes is given by the fact that in case of dealing with higher-order probability modeling, multiple sets of VLC tables may be necessitated. Arithmetic coding, on the other hand, while being substantially more complex than VLC, offers the advantage of a more consistent and adequate handling when coping with adaptive and higher-order probability modeling as well as with the case of highly skewed probability distributions. Actually, this characteristic basically results from the fact that arithmetic coding provides a mechanism, at least conceptually, to map any given value of probability estimate in a more or less direct way to a portion of the resulting codeword. Being provided with such an interface, arithmetic coding allows for a clean separation between the tasks of probability modeling and probability estimation, on the one hand, and the actual entropy coding, i.e., mapping of a symbols to codewords, on the other hand.
For the sake of clarity, we will restrict the exposition in the following to the case of a binary alphabet, although conceptually, the very basic methods of this invention also apply to the general case of an m-ary alphabet with m>2. The probability distribution of symbols with values in a binary alphabet can be described by a single parameter p=pLPS, where the useful distinction is made between the so-called less probable symbol (LPS) with probability estimates pLPS≦0.5 and the more probable symbol (MPS) with pMPS=1−pLPS. Thus, in the general case we have 0<pLPS≦0.5, where in practical cases, the probability estimate is often bounded from below by a minimum probability value pmin such that pmin≦pLPS.